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This experiment investigates the capability of Synthetic Vision Systems (SVS) to provide significant 
situation awareness in terminal area operations, specifically in low visibility conditions. The use of a 
Head-Up Display (HUD) and Head-Down Displays (HDD) with SVS is contrasted to baseline standard 
head down displays in terms of induced workload and pilot behavior in 1400 RVR visibility levels. 
Variances across performance and pilot behavior were reviewed for acceptability when using HUD or 
HDD with SVS under reduced minimums to acquire the necessary visual components to continue to land. 
The data suggest superior performance for HUD implementations. Improved attentional behavior is also 
suggested for HDD implementations of SVS for low- visibility approach and landing operations. 


INTRODUCTION 

NASA Langley Research Center (NASA LaRC) and the 
Department of Transportation/Federal Aviation 
Administration (DOT/FAA) are conducting collaborative 
research to ensure effective technology development and 
regulatory guidance for Synthetic Vision Systems/Enhanced 
Flight Vision Systems (SVS/EFVS) advanced cockpit vision 
technologies in Next Generation Air Transportation System 
(NextGen) operations. These technologies have the potential 
to provide an additional margin of safety and aircrew 
performance to enable operational improvements for low 
visibility surface, arrival, and departure operations in the 
terminal environment with equivalent efficiency as visual 
operations. 

Synthetic Vision is the display of a “computer-generated 
image of the external scene topography from the perspective 
of the flight deck that is derived from aircraft attitude, high- 
precision navigation solution, and database of terrain, 
obstacles and relevant cultural features” (FAA 2011). This 
technology has been shown to increase the level of terrain and 
situation awareness (SA) by displaying intuitive visual 
information that mimics the out-the-window visuals when 
none is actually available due to poor visibility. The outcome 
is documented improvement of safety against controlled- 
fhght-into-terrain and possibly, loss-of-control accidents 
(Prinzel, et al., 2000; Bateman, 2010). 

SVS technology exists today and is certified for general 
aviation aircraft and numerous business jets such as the 
Gulfstream V (Freeman 2002); however, despite significant 
improvements in terrain and situation awareness, at present, 
no additional ‘operational credit’ is provided from its 
installation or use. 


The Current Study 

A fixed-base simulation experiment was conducted to 
investigate if a lower decision height or reduced visibility 
minima is warranted by the use of SVS (Figure 1), thus 
providing ‘operational credit’. The experiment was designed 
around the current operating regulations for landing under 
Instrument Flight Rules, as defined under 14 Code of Federal 
Regulations (CFR) §91.175 and following the precedent 
established under FAA Order 8400.13 (“Procedures for the 
Evaluation and Approval of Facilities for Special 
Authorization Category I Operations and All Category II and 
III Operations”). The premise was that integrated terrain and 
symbology presentation of the SVS merits operational credit 
since it: 

1) Reduces flight technical error and promotes a 
stabilized approach; thus, ensuring that the aircraft 
will be in the proper position at the end of the 
instrument segment with minimal transition from 
which to complete the flare and landing; 

2) Eases the pilot’s transition from the instrument to the 
visual flight segment, by improving the pilot’s ability 
to correctly recognize and identify the required visual 
references as per 14 CFR §91.175 because the 
intuitive visual-like SVS information promotes 
efficient out-the-window visual search. 

A key element of this work is quantifying the pilots’ 
distribution of attention and their workload and ability to 
transition from the instrument segment to the visual segment 
during an instrument approach to landing in low visibility 
conditions. This work also addressed the influence of head-up 
display (HUD) or head-down display (HDD) implementations. 
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METHOD 

Flight Test Equipment 

Simulation testing took place in NASA LaRC’s Research 
Flight Deck (RFD), modeled after current state-of-the-art 
aircraft with four 10.5” by 13.25” liquid crystal color displays 
(Figure 2). The RFD is a full-mission simulator with 
hydraulic-actuated side-stick control inceptors. The aircraft 
model was a Boeing 757-200. 
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Figure 1. Proposed SVS Ops Change 



Figure 2. NASA Research Flight Deck 


Altitude call-outs were given over the flight deck audio 
system, including “500 feet”, “approaching minimums” and 
“minimums”, referencing ownship to height above threshold 
(HATh). Additionally, call-outs of radar altitude starting at 
100 ft in 10 ft increments to touchdown were also included. 


The out-the-window (OTW) visual scene is generated by an 
Evans and Sutherland image generator and collimated 
projection system. The OTW field-of-view is 200 deg. 
horizontal (H) by 40 deg. vertical (V) at 26 pixels per deg. 

The four HDDs were populated with the Pilot Flying (PF), 
Primary Flight Display (PFD), the PF Navigation Display 
(ND), the Pilot Monitoring (PM) ND, and the PM PFD. The 
content on these displays was experimentally dependent, but 
followed standard PFD/ND design protocol. 

The left (PF) seat of the RFD is equipped with a Flight 
Dynamics HGS-4000 HUD, collimated and subtending a 26° 
H by 21° V field-of-view. The HUD projects imagery in 
stroke and raster format for symbology and SVS imagery, 
respectively. Pilots were able to “declutter” the stroke and 
raster imagery independently as desired using a four-button 
castle switch located on the sidestick inceptor. 

A 4-camera Smart Eye™ head and eye tracking system was 
installed in the left seat of the RFD to capture quantitative data 
of PF head and eye movement. 

Simulator Database 

The test was simulated at Chicago O’ Hare International 
Airport. Weather effects were experimentally varied, but for 
this analysis/paper, the weather and associated visibility, 
measured by Runway Visibility Range (RVR), were held 
constant. Calligraphic projection simulated a Medium 
intensity Approach Lighting System with runway alignment 
indicator. Runway centerline (CL) and touchdown zone 
(TDZ) lights were also experimentally varied. 


Experimental Design 

The experiment investigated, among other objectives, three 
comparisons of SVS application and their effect on pilot 
attention during transition to visual acquisition on short final: 
1) SVS location (HUD vs. HDD), 2) SVS equipage (SVS 
HDD vs. No-SVS HDD, Figure 3), and 3) Effect of single 
versus dual crew operations. Comparative “baseline” 
conditions were those without SVS imagery. The experiment 
matrix is shown in Table 1 with the comparisons using the 
numbering above. 

The single versus dual crew operations conditions were run to 
assess the PF’s attentional, behavioral, and visual scanning 
with and without a PM assisting with verbal observations and 
call-outs, especially in the instrument-to-visual transition. The 
influence of SVS on this condition can only be inferred since 
baseline runs were not flown in single operations. 

HUD presentation of SVS was shown in green raster on the 
combiner glass of the HUD, projected to be conformal to the 
environment from the pilot eye point (Figure 3) with overlaid 
symbology. 

Variance in CL and TDZ lighting was included as an even 
distribution across conditions. 

This research utilized 1 1 scenarios presented to each crew to 
address all required configuration comparisons, 8 crewed 
operations and three single pilot operations. A total of four 
HUD scenarios were presented in the crewed operation, with 
two baseline HUD and two SVS HUD scenarios (one each 
with the TDZ and CL lights varied). Four HDD scenarios 
were presented, two with baseline and two SVS configurations 
scenarios (one each with the TDZ and CL lights varied). 



Single pilot scenarios evaluated only the SVS configuration, 
totaling three scenarios. 

Initial conditions for all scenarios positioned the aircraft lined 
up for the approach on final inside the final approach fix. 
Four wind profiles were generated to provide headwind, 
tailwind and crosswind elements no greater than 15 kts. All 
scenarios were presented with an OTW visibility of 1400 ft 
and a decision height of 150 HATh. 



Figure 3. HUD with SVS and Symbology 



Figure 4. Baseline (left) and SVS (right) HDD Configurations 


Table 1. Experiment Matrix 


Vision 

Display System 

Crewed 

1400 RVR 

Single 

1400 RVR 

None 

1 


HUD sv 

1 


None 

1,2 


HDD sv 

1,2,3 

3 


minimize crew coordination issues that might otherwise arise 
due to variance in Standard Operating Procedures and Crew 
Resource Management training. Pilots were trained in a pre- 
flight briefing and in the simulator for the various display 
configurations and crew procedures for SVS and EVS 
operations under the evaluation criteria. 

Results 

Pilot Workload 

A standardized Air Force Flight Test Center workload scale 
questionnaire was administered after each scenario (Gawron, 
2000 ). 

In the first statistical comparison of SVS location (HUD 
versus HDD), there was a significant difference (F(l,68)=5.71, 
p= 0.020) indicating the utilization of a HUD effectively 
reduces the PF workload from 3.68 (HDD) to 3.13 (HUD). 

The second statistical comparison evaluating SVS equipage 
indicated there was no significant effect on PF workload. SVS 
on a HDD yielded an average 3.64 relative to a baseline 
condition HDD (No SVS) average of 3.74. 

The third comparison - effect of crewed versus single pilot 
operations - no statistically significant differences were found. 
Crewed operations yielded an average workload of 3.64 and 
single pilot operation yielded a slightly increased average 
workload of 3.78. 

Evaluation Pilot Behavior 

Pilot attention and visual behavior was inferred by the head 
and eye tracking data to determine when pilots transitioned 
from instrumentation to OTW visual to acquire the runway 
environment to continue to landing. Metrics used to evaluate 
pilot behavior included; head-up percentage, transition count 
between OTW and head-down. For HDD concepts only, the 
transition glance - the altitude at which the PF first looked 
OTW - and the “full transition” OTW HATh altitude was also 
determined to evaluate variance in pilots’ visual acquisition 
behavior. 

Pilot eye and head tracking data was broken into five 
segments; three in the approach phase (Instrument, 
Instrument- to- visual, and Visual segments) and two in the 
landing phase (Flare and Landing, including roll-out, 
segments), to provide logical contrast across the phases of 
final approach and landing (Figure5). 


Evaluation Pilots 

24 pilots making up 12 crews were used as evaluation pilots. 
Each pilot held an Airline Transport Pilot rating and had at 
least 100 hours pilot- in-command HUD experience. 
Preference was given to pilots with operational experience 
with SVS and EVS. Pilots were paired by current employer to 


SVS Location Comparison (HUD vs. HDD) 

Eye/head tracking results showed significant effects on head- 
up percentage across all three approach segments 
(F(ll,l)=34.13, /?<0.000), no effect between locations during 
the flare segment, and significance in the landing segment 
(F(ll,l)=6.41,/?=0.028). HDD concept findings indicate that 
pilots during the visual segment still remain head-down on 






average 30% of the time, scanning information available on 
the PFD. The landing segment saw statistical significance, 
however, operational significance is weak, with conditions 
varying between approximately 85-95% head up. 


150 ft HATh 
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Figure 5. Eye Tracking Analysis Segments 

Transition count analysis indicated similar significance 
findings to that of the head-up percentage, with the exception 
of the landing segment where no statistical significance 
existed between location conditions. However, the landing 
segment did show significance (F(l 1,1)=5.85, p= 0.034) with 
the presence of TDZ/CL lights driving up the number of 
transitions made regardless of SVS location. 

SVS HUD vs. Baseline HUD Comparison 

Head-up percentage analysis results show no significant 
difference in pilot attentional behavior between SVS and 
Baseline display concepts presented on the HUD across all 
segments of approach and landing. Head-up percentage 
averages ranged between 87% and 98% for the Baseline HUD 
concept, and 91% and 99% for the SVS HUD concept. 

Transition count analysis indicates no significant effect 
between SVS and Baseline display concepts presented on the 
HUD across all segments of approach and landing. Transition 
count averages ranged between 0.13 and 5.5 for the Baseline 
HUD concept, and 0.12 and 4.4 for the SVS HUD concept. 

SVS HDD vs. Baseline HDD Comparison 

Head-up percentage analysis results show no significant 
effects during any segment with the exception of the visual 
segment (F=(ll,l)=7.39, /?=0.020), in which SVS drove pilot 
attention out-the-window 71% compared to the baseline 
average of 58% during this segment. 

Transition count analysis indicates no significant effects 
between SVS HDD and Baseline HDD concepts across all of 
the flight segments. This data suggests the presence of SVS 
do not alter pilot OTW transition behavior relative to the 
baseline HDD concept. TDZ/CL lights did show a significant 
effect during the landing segment (F=(l 1,1)=78.23, /?<0.000), 
again suggesting the presence of TDZ/CL lights drive up the 
number of OTW transitions regardless of HDD concept. 

SVS HDD Crewed vs. Single Comparison 

Head-up percentage analysis results show no significant 
effects during any segment with the exception of the landing 
segment (F(l 1,1)=15.69, p= 0.002), during which single crew 


operation drives pilot attention inside the cockpit 25% of the 
time relative to crewed operation where PF attention was 
inside the cockpit 12% of the time, on average. 

Transition count analysis indicates similar results to that of the 
head-up analysis, showing only the landing segment to contain 
significant difference between crewed and single pilot 
operations (F(ll,l)=20.42, /?=0.001). Single pilot operation 
drives the pilot to transition between HDD and OTW on 
average approximately 9 times relative to the crewed 
operations average of 4.5. 

SVS HDD vs. Baseline HDD Concept Transition HATh 

Pilot initial glance transition data showed statistical 
significance between the SVS HDD and Baseline HDD 
concepts (F(l,70)=6.98, /?=0.010), indicating SVS concepts 
keep pilot attention inside the flight deck until closer to the 
DH. Baseline initial glance transition occurred at 377 ft while 
SVS initial glance transitions occurred at 266 ft, on average. 

The full transition analysis showed no significant difference 
between Baseline and SVS concepts, with values of 192 ft and 
164 ft, respectively. While not statistically significant, these 
values still show the same trend observed in the initial glance 
data, suggesting SVS maintains attention inside the flight deck 
until closer to the DH. 

There was no significant difference between crewed and 
single pilot operations in initial glance HATh with both 
occurring above the DH on average at 266 ft and 270 ft, 
respectively. Full visual transition OTW was not statistically 
significant; however, data shows single pilot full transition 
does not occur until below the DH on average (130 ft), while 
crewed operations fully transitioned OTW above the DH - at 
164 ft on average. 

Pilot Comments 

Pilot comments were collected during and after the experiment 
to collect professional insight into use of the systems and their 
impact on performance, workload, and acceptability/usability. 
Pilots ranked their low-visibility operations display concept 
preferences as follow: 1. SVS HUD, 2. Baseline HUD, 3. SVS 
HDD, 4. Baseline HDD. These results suggest the display 
location to be paramount in selection criteria, followed by 
display concept, with SVS being more preferable than 
baseline. These preferences were statistically significant 
across the display concepts (j^(4)=3L145,/?<0.001). 

Discussion 

These analyses are intended to quantify the pilots’ distribution 
of attention and their workload and ability to transition from 
the instrument segment to the visual segment during an 
instrument approach to landing and rollout in low visibility 
conditions. This work also addressed the influence of HUD or 
HDD implementations of SVS, and crewed or single pilot 
operations while utilizing SVS. Also considered is the 


influence of the runway environment (from the presence or 
absence of TDZ and CL lights). 

The results of the analyses, as expected, provide a strong 
argument supporting the utilization of a HUD. It is not only 
preferred by pilots, but through quantitative comparison, it is 
effective in keeping pilot attention out-the-window and 
minimizing the time spent transitioning between instruments 
on the HDD and OTW. This behavior is particularly 
prominent in the instrument-to-visual and visual flight 
segments. The data indicates that, on average, the PF went 
head-down 30% of the time even in the visual segment. The 
benefit of HUD usage is further supported by a reduction in 
reported pilot workload. 

No significant attentional variance was found when SVS was 
shown on the HUD. Future work will assess HUD clutter and 
masking effects. These influences are of critical importance in 
determining the safety impact of allowing pilots to utilize 
HUD-projected SVS. Reduced minima effectively reduce the 
time between decision height and touchdown, thus, increasing 
the criticality of acquiring the runway environment. 

For HDD concepts, introducing SVS resulted in increasing the 
time spent OTW during the visual segment of flight and no 
significant difference in OTW-HDD transition counts when 
compared to the baseline. These results suggest the use of 
SVS provides an increased SA to pilots during the time critical 
short final in low visibility, allowing them to quickly glance at 
the HDD and obtain the necessary information to then again 
transition OTW to acquire the runway environment. SVS 
HDDs kept pilot attention inside the flight deck until much 
closer to the DH, possibly suggesting the SVS imagery affords 
the pilot a more decisive transition cue at the published DH 
when visual acquisition of the runway environment is 
increasingly likely in poor visibility conditions. There was no 
significant variance in workload reported between SVS and 
Baseline HDD conditions signifying pilots easily adapt to use 
of the SVS display concepts. 

Comparing SVS concepts for crewed and single pilot 
operations, showed the influence of the PM’s assistance. In the 
landing and roll-out, single pilot operations require the pilot to 
transition twice as frequently between OTW and HDD to 
achieve the proper SA to complete the task of exiting the 
runway at the proper turnoff. This is reflected in a 10% 
reduction of head up time when the pilot in command should 
maximize visual attention OTW to scan for surface traffic. 
The PF was drawn into the cockpit to reference a HDD-airport 
moving map for awareness of positioning on the runway and 
the location of the expected taxi exit. In crewed operations, 
the PM provided verbal assistance and direction. Transition 
HATh analysis shows that single pilot operations were slower 
in fully transitioning attention OTW compared to crewed 
operations. A full transition is indicative of the runway 
environment being visually acquired allowing pilots to 
complete the visual approach. These findings may suggest 
single pilots were more workload saturated attempting to 
monitor the approaching DH as well as the precision guidance 


than when flying as a crew. However, these quantitative data 
disassociate from the insignificant differences in subjective PF 
workload ratings between crewed and single pilot operations. 
This leads to the argument that while pilots may not feel a 
difference in perceived workload, operationally there is a 
significant impact. These results strongly suggest that the 
presence of a crew member significantly effects pilot 
transition behavior by reducing the number of times a pilot is 
driven to reference information inside the flight deck while 
maintaining proper safe attention out-the-window. 

Acknowledgements 

This work was conducted through joint sponsorship of NASA, 
under the Aviation Safety Program, Vehicle Systems Safety 
Technology project, and the FAA under the Human Factors 
Research and Development Project for NextGen. The support 
of Mr. Randall E. Bailey, Dr. Steven D. Young, NASA, Dr. 
Tom McCloy, Ms. Terry Stubblefield, and Mr. Dan Herschler, 
FAA, is gratefully appreciated. 

References 

Bailey, R. E., L. J. Kramer, and S. P. Williams. ’’Enhanced 
Vision for All-Weather Operations Under NextGen.’’ SPIE 
Defense Security and Sensing Conference Proceedings. 
Orlando, FL: SPIE, 2010. vol. 7689. 

Bateman, D. “Some Thoughts on Reducing the Risk of 
Aircraft Loss of Control,” paper presented at the 2010 AIAA 
Guidance, Navigation, and Control Conference, AIAA Paper 
No. AIAA 2010-8139, Toronto, Canada, 2-5 August 2010. 

FAA. Code of Federal Regulations. February 15, 2011. 

FAA. Procedures for the Evaluation and Approval of 
Facilities for Special Authorization Category I Operations and 
All Category II and III Operations. October 22, 2009. 

Freeman, G. ’’Gulfstream Enhanced Vision System.” 
Symposium Proceedings - Society of Experimental Test Pilots. 
Society of Experimental Test Pilots, 2002. 154-168. 

Gawron, V. J. “Human Performance Measures Handbook” 
Lawerence Erlbaum and Assoicates, New Jersey. 2000. 

Hemm, R., G. Shapiro, D. Lee, J. Gribko, and B. Glaser. 
Beneift Estimates of Terminal Area Productivity Program 
Technologies. Final Report, Hampton, VA: NASA Langley 
Research Center, 1999. 

Prinzel, L J., J.R. Comstock, D. Elliot, L.J. Kramer, L.B. 

Glaab, and R.V. Parrish. ’’The NASA Synthetic Vision 
Systems research project.” Proceedings of the 4 th Human 
Performance, Situation Awareness, and Automation: User- 
Centered Design for the New Millennium Conference 
Proceedings. Savannah, GA: North Carolina State University, 
SA Technologies, Mississippi State University, 2000. 359. 


